Skip to content

Allow to build LexBuffer from array without copying. - #4882

Merged
KevinRansom merged 3 commits into
dotnet:masterfrom
smoothdeveloper:LexBuffer-no-array-copy-when-built-from-chars
May 11, 2018
Merged

KevinRansom merged 3 commits into
dotnet:masterfrom
smoothdeveloper:LexBuffer-no-array-copy-when-built-from-chars

Conversation

@smoothdeveloper

Copy link
Copy Markdown
Contributor

All usages of FromChars seem to be taking a throw away array that caller don't keep a reference, usages found:

src\fsharp\lex.fsl(155)
src\fsharp\UnicodeLexing.fs(18)
src\fsharp\UnicodeLexing.fs(68)
src\fsharp\FSharp.Compiler.Private\lex.fs(154)
tests\FSharp.Compiler.UnitTests\HashIfExpression.fs(71)

All usages of FromChars seem to be taking a throw away array that caller don't keep a reference, usages found:

src\fsharp\lex.fsl(155)
src\fsharp\UnicodeLexing.fs(18)
src\fsharp\UnicodeLexing.fs(68)
src\fsharp\FSharp.Compiler.Private\lex.fs(154)
tests\FSharp.Compiler.UnitTests\HashIfExpression.fs(71)
Comment thread src/utils/prim-lexing.fs

// A full type signature is required on this method because it is used at more specific types within its own scope
static member FromArray (s: 'Char[]) : LexBuffer<'Char> =
// Important: This method takes ownership of the array

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ask the borrow checker ;-)

Comment thread src/utils/prim-lexing.fs
// Important: This method takes ownership of the array
static member FromArrayNoCopy (buffer: 'Char[]) : LexBuffer<'Char> =
let lexBuffer = new LexBuffer<'Char>(fun _ -> ())
let buffer = Array.copy s

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove the ; while you are touching this

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, were left due to oversight.

Comment thread src/utils/prim-lexing.fsi
@@ -65,6 +65,7 @@ type internal LexBuffer<'Char> =
member IsPastEndOfStream: bool with get,set

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FromArryNoCopy should go here

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notice that FromArray isn't exposed in this file, this is why I haven't added the extra method.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok thanks

Comment thread src/utils/prim-lexing.fs Outdated
lexBuffer

static member FromChars (arr:char[]) = LexBuffer.FromArray arr
// A full type signature is required on this method because it is used at more specific types within its own scope

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the comment about full signature can go.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Waiting for further confirmation, if I should remove all the same comments on the two other methods as well.

Notice that there are dubious comments here:

https://github.com/Microsoft/visualfsharp/blob/75e435e74b81bb47c4a0ced11be0d7560d2a66ea/src/fsharp/UnicodeLexing.fs#L35-L40

It doesn't look like those comments apply as the stream is totally handled within the method and we just return the LexBuffer object.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah those comments look old

@smoothdeveloper

Copy link
Copy Markdown
Contributor Author

I have no clue if this failing test is broken because of the change:

=> FSharp.Core.UnitTests.FSharp_Core.Type_Forwarding.TypeForwardingModule.TypeForwarding
currentRuntimeVersion = v4.0.30319; currentFSharpCoreTargetRuntime=v4.0.30319 tupleAssemblyName=mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
-----------------------------------------------------------------
-----------------------------------------------------------------
Error: Running tests net40-coreunit failed, see logs above -- FAILED
-----------------------------------------------------------------

If it is, I'd somewhat expect more tests to be broken, not just one.

@dsyme

dsyme commented May 11, 2018

Copy link
Copy Markdown
Contributor

The failing test is this one:

1) Failed : FSharp.Core.UnitTests.FSharp_Core.Microsoft_FSharp_Control.AsyncModule.SleepContinuations
  Expected: 0
  But was:  1

@dsyme

dsyme commented May 11, 2018

Copy link
Copy Markdown
Contributor

@smoothdeveloper I think that may have been a transient failure due to timing. I was worried the recent control.fs cleanup had caused it but the test is passing on my machine

@dsyme

dsyme commented May 11, 2018

Copy link
Copy Markdown
Contributor

@dotnet-bot test Windows_NT Release_ci_part2 Build please

@KevinRansom
KevinRansom merged commit 142c139 into dotnet:master May 11, 2018
T-Gro pushed a commit that referenced this pull request Jun 12, 2026
* Allow to build LexBuffer from array without copying.

All usages of FromChars seem to be taking a throw away array that caller don't keep a reference, usages found:

src\fsharp\lex.fsl(155)
src\fsharp\UnicodeLexing.fs(18)
src\fsharp\UnicodeLexing.fs(68)
src\fsharp\FSharp.Compiler.Private\lex.fs(154)
tests\FSharp.Compiler.UnitTests\HashIfExpression.fs(71)

* remove spurious semicolons in FromArrayNoCopy

* removing spurious comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants